Stopping Criteria for the Iterative Solution of Linear Least Squares

We explain an interesting property of minimum residual iterative methods for the solution of the linear least squares (LS) problem. Our analysis demonstrates that the stopping criteria commonly used with these methods can in some situations be too conservative, causing any chosen method to perform too many iterations or even fail to detect that an acceptable iterate has been obtained. We propose a less conservative criterion to determine whether a given iterate is an acceptable LS solution. This is merely a sufficient condition, but it approaches a necessary condition in the limit as the given iterate approaches the exact LS solution. We also propose a necessary and sufficient condition to determine whether a given approximate LS solution is an acceptable LS solution, based on recent results on backward perturbation analysis of the LS problem. Although both of the above new conditions use quantities that are too expensive to compute in practical situations, we suggest potential approaches for estimating some of these quantities efficiently. We illustrate our results with several numerical examples.